Community Detection In Graphs Research Articles

Heterogeneous graph community detection method based on K-nearest neighbor graph neural network

Traditional community detection models either ignore the feature space information and require a large amount of domain knowledge to define the meta-paths manually, or fail to distinguish the importance of different meta-paths. To overcome these limitations, we propose a novel heterogeneous graph community detection method (called KGNN_HCD, heterogeneous graph Community Detection method based on K-nearest neighbor Graph Neural Network). Firstly, the similarity matrix is generated to construct the topological structure of K-nearest neighbor graph; secondly, the meta-path information matrix is generated using a meta-path transformation layer (Mp-Trans Layer) by adding weighted convolution; finally, a graph convolutional network (GCN) is used to learn high-quality node representation, and the k-means algorithm is adopted on node embeddings to detect the community structure. We perform extensive experiments and on three heterogeneous datasets, ACM, DBLP and IMDB, and we consider as competitors 11 community detection methods such as CP-GNN and GTN. The experimental results show that the proposed KGNN_HCD method improves 2.54% and 2.56% on the ACM dataset, 2.59% and 1.47% on the DBLP dataset, and 1.22% and 1.67% on the IMDB dataset for both NMI and ARI. Experiments findings suggest that the proposed KGNN_HCD method is reasonable and effective, and KGNN_HCD can be applied to complex network classification and clustering tasks.

Algorithm xxx: PyGenStability, a multiscale community detection with generalized Markov Stability

We present PyGenStability, a general-use Python software package that provides a suite of analysis and visualisation tools for unsupervised multiscale community detection in graphs. PyGenStability finds optimized partitions of a graph at different levels of resolution by maximizing the generalized Markov Stability quality function with the Louvain or Leiden algorithms. The package includes automatic detection of robust graph partitions and allows the flexibility to choose quality functions for weighted undirected, directed and signed graphs, and to include other user-defined quality functions.

Kernel Based Dual-Channel Attributed Graph Community Detection

Kernel Based Dual-Channel Attributed Graph Community Detection

Intelligent information extraction from government on-site inspection reports of construction projects: A graph-based text mining approach

Government inspection reports detail unsafe acts and conditions that arise on construction sites, especially front-line managers’ non-compliance practices, which are hardly identified during self-inspections. Such information serves as a valuable learning source for better construction management. However, non-compliance issue records in inspection reports are typically stored in unstructured text formats, making data analysis challenging. In response, an intelligent text mining framework integrating graph analysis and visualization is presented. The proposed framework comprises data collection and preprocessing and three levels of text analysis: word, sentence, and document. The main tasks of the word-level analysis include (1) extracting keywords using KeyBERT and (2) identifying non-compliance issue types based on community detection in a keyword co-occurrence graph. The sentence-level analysis is performed to automatically classify text data from inspection reports by determining the degree of similarity between texts and communities and assigning the most similar community to each text. The document-level analysis aims to identify the interrelations between various non-compliance issues through association rule mining and a community interaction network. The framework is validated by a total of 6,153 text data featuring non-compliance issues from 322 government on-site inspection reports in Shanghai, China. The results demonstrate that the critical word-level features of non-compliance issues can be accurately identified using KeyBert, which outperforms other state-of-the-art methods. Our approach can also automate the development of a data-driven taxonomy for non-compliance issues and the classification of the corresponding records, requiring less manual intervention than conventional text classification models.

Distributed Genetic Algorithm for Community Detection in Large Graphs with a Parallel Fuzzy Cognitive Map for Focal Node Identification

This study addresses the importance of focal nodes in understanding the structural composition of networks. To identify these crucial nodes, a novel technique based on parallel Fuzzy Cognitive Maps (FCMs) is proposed. By utilising the focal nodes produced by the parallel FCMs, the algorithm efficiently creates initial clusters within the population. The community discovery process is accelerated through a distributed genetic algorithm that leverages the focal nodes obtained from the parallel FCM. This approach mitigates the randomness of the algorithm, addressing the limitations of the random population selection commonly found in genetic algorithms. The proposed algorithm improves the performance of the genetic algorithm by enabling informed decision making and forming a better initial population. This enhancement leads to improved convergence and overall algorithm performance. Furthermore, as graph sizes grow, traditional algorithms struggle to handle the increased complexity. To address this challenge, distributed algorithms are necessary for effectively managing larger data sizes and complexity. The proposed method is evaluated on diverse benchmark networks, encompassing both weighted and unweighted networks. The results demonstrate the superior scalability and performance of the proposed approach compared to the existing state-of-the-art methods.

LazyFox: fast and parallelized overlapping community detection in large graphs.

The detection of communities in graph datasets provides insight about a graph's underlying structure and is an important tool for various domains such as social sciences, marketing, traffic forecast, and drug discovery. While most existing algorithms provide fast approaches for community detection, their results usually contain strictly separated communities. However, most datasets would semantically allow for or even require overlapping communities that can only be determined at much higher computational cost. We build on an efficient algorithm, Fox, that detects such overlapping communities. Fox measures the closeness of a node to a community by approximating the count of triangles which that node forms with that community. We propose LazyFox, a multi-threaded adaptation of the Fox algorithm, which provides even faster detection without an impact on community quality. This allows for the analyses of significantly larger and more complex datasets. LazyFox enables overlapping community detection on complex graph datasets with millions of nodes and billions of edges in days instead of weeks. As part of this work, LazyFox's implementation was published and is available as a tool under an MIT licence at https://github.com/TimGarrels/LazyFox.

Explainable AI in big data intelligence of community detection for digitalization e-healthcare services

Recommender Systems are designed to analysis the available data in the system to predict user’s desires and provide appropriate personalized suggestions to each user that suits their interests. In this paper, we have developed an explainable medical recommender system that uses graph concepts to provide an interpretable approach to medical data. The presented approach is based on community detection algorithms. It forms a graph between the users based on their similarity scores. Individuals with common interests are then grouped using graph community detection algorithms. Two community detection algorithms have been applied on the graphs of users and physicians in our medical recommender system. The results of applying two community detection algorithms are then used to address the cold start problem. We have identified the most influential users using a graph-based technique that finds the overlapping communities. We claim that using the overlapping graph of communities to address cold start problem will enhance the accuracy of the recommendations. Weighting or voting systems are also applied on the selected users to give feedback to potential consumers where there are n different options in a cluster. The similarity score of the users in the overlapping communities has been used to weight the final recommendation. The accuracy of recommended services depends on the proper selection of target populations. The proposed approach outperforms the use of each one of the community detections separately. The accuracy and precision of the proposed method are 93.06 and 88.34, which exceed the highest achieved accuracy in the literature.

Community Detection With Known, Unknown, or Partially Known Auxiliary Latent Variables

Empirical observations suggest that in practice, community membership does not completely explain the dependency between the edges of an observation graph. The residual dependence of the graph edges are modeled in this paper, to first order, by auxiliary node latent variables that affect the statistics of the graph edges but <italic xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">carry no information about the communities of interest.</i> We then study community detection in graphs obeying the stochastic block model and censored block model with auxiliary latent variables. We analyze the conditions for exact recovery when these auxiliary latent variables are unknown, representing unknown nuisance parameters or model mismatch. We also analyze exact recovery when these secondary latent variables have been either fully or partially revealed. Finally, we propose a semidefinite programming algorithm for recovering the desired labels when the secondary labels are either known or unknown. We show that exact recovery is possible by semidefinite programming down to the respective maximum likelihood exact recovery threshold.

On Spectral Algorithms for Community Detection in Stochastic Blockmodel Graphs With Vertex Covariates

In network inference applications, it is often desirable to detect community structure, namely to cluster vertices into groups, or blocks, according to some measure of similarity. Beyond mere adjacency matrices, many real networks also involve vertex covariates that carry key information about underlying block structure in graphs. To assess the effects of such covariates on block recovery, we present a comparative analysis of two model-based spectral algorithms for clustering vertices in stochastic blockmodel graphs with vertex covariates. The first algorithm uses only the adjacency matrix, and directly estimates the block assignments. The second algorithm incorporates both the adjacency matrix and the vertex covariates into the estimation of block assignments, and moreover quantifies the explicit impact of the vertex covariates on the resulting estimate of the block assignments. We employ Chernoff information to analytically compare the algorithms' performance and derive the information-theoretic Chernoff ratio for certain models of interest. Analytic results and simulations suggest that the second algorithm is often preferred: we can often better estimate the induced block assignments by first estimating the effect of vertex covariates. In addition, real data examples also indicate that the second algorithm has the advantages of revealing underlying block structure and taking observed vertex heterogeneity into account in real applications. Our findings emphasize the importance of distinguishing between observed and unobserved factors that can affect block structure in graphs.

Clustering biological sequences with dynamic sequence similarity threshold

BackgroundBiological sequence clustering is a complicated data clustering problem owing to the high computation costs incurred for pairwise sequence distance calculations through sequence alignments, as well as difficulties in determining parameters for deriving robust clusters. While current approaches are successful in reducing the number of sequence alignments performed, the generated clusters are based on a single sequence identity threshold applied to every cluster. Poor choices of this identity threshold would thus lead to low quality clusters. There is however little support provided to users in selecting thresholds that are well matched with the input sequences.ResultsWe present a novel sequence clustering approach called ALFATClust that exploits rapid pairwise alignment-free sequence distance calculations and community detection in graph for clusters generation. Instead of a single threshold applied to every generated cluster, ALFATClust is capable of dynamically determining the cut-off threshold for each individual cluster by considering both cluster separation and intra-cluster sequence similarity. Benchmarking analysis shows that ALFATClust generally outperforms existing approaches by simultaneously maintaining cluster robustness and substantial cluster separation for the benchmark datasets. The software also provides an evaluation report for verifying the quality of the non-singleton clusters obtained.ConclusionsALFATClust is able to generate sequence clusters having high intra-cluster sequence similarity and substantial separation between clusters without having users to decide precise similarity cut-off thresholds.

Leveraging graph algorithms to speed up the annotation of large rhymed corpora

Abstract Rhyming patterns play a crucial role in the phonological reconstruction of earlier stages of Chinese. The past few years have seen the emergence of the use of graphs to model rhyming patterns, notably with List’s (2016) proposal to use graph community detection as a way to go beyond the limits of the link-and-bind method and test new hypotheses regarding phonological reconstruction. List’s approach requires the existence of a rhyme-annotated corpus; such corpora are rare and prohibitively expensive to produce. The present paper solves this problem by introducing several strategies to automate annotation. Among others, the main contribution is the use of graph community detection itself to build an automatic annotator. This annotator requires no previous annotation, no knowledge of phonology, and automatically adapts to corpora of different periods by learning their rhyme categories. Through a series of case studies, we demonstrate the viability of the approach in quickly annotating hundreds of thousands of poems with high accuracy.

A security detection approach based on autonomy-oriented user sensor in social recommendation network

User social network-based recommender system has achieved significant performance in current recommendation fields. However, the characteristic of openness brings great hidden dangers to the security of recommender systems. Shilling attackers can change the recommendations by foraging user relationships. Most shilling attack detection approaches depend on the explicit user historical data to locate shilling attackers. Some important features such as information propagation and social feedback of users in social networks have not been noticed. We propose a security detection method based on autonomy-oriented user sensor (AOUSD) to identify shilling attackers. Specifically, (1) the user is simulated as a social sensor with autonomous capabilities, (2) the user interaction model is built based on information propagation, information feedback and information disappearance mechanisms of social sensors, and a user dynamic knowledge graph is formed by considering the variable time function, (3) hierarchical clustering method is used to generate preliminary suspicious candidate groups and graph community detection clustering method is applied on the dynamic knowledge graph to detect the attackers. Then, AOUSD is first simulated on NetLogo and it is compared with other algorithms based on the Amazon data. The results prove the advantages of AOUSD in the efficiency and accuracy on shilling attack detection.

Scalable distributed Louvain algorithm for community detection in large graphs

Scalable distributed Louvain algorithm for community detection in large graphs

LDPCD: A Novel Method for Locally Differentially Private Community Detection.

As one of the cores of data analysis in large social networks, community detection has become a hot research topic in recent years. However, user's real social relationship may be at risk of privacy leakage and threatened by inference attacks because of the semitrusted server. As a result, community detection in social graphs under local differential privacy has gradually aroused the interest of industry and academia. On the one hand, the distortion of user's real data caused by existing privacy-preserving mechanisms can have a serious impact on the mining process of densely connected local graph structure, resulting in low utility of the final community division. On the other hand, private community detection requires to use the results of multiple user-server interactions to adjust user's partition, which inevitably leads to excessive allocation of privacy budget and large error of perturbed data. For these reasons, a new community detection method based on the local differential privacy model (named LDPCD) is proposed in this paper. Due to the introduction of truncated Laplace mechanism, the accuracy of user perturbation data is improved. In addition, the community divisive algorithm based on extremal optimization (EO) is also refined to reduce the number of interactions between users and the server. Thus, the total privacy overhead is reduced and strong privacy protection is guaranteed. Finally, LDPCD is applied in two commonly used real-world datasets, and its advantage is experimentally validated compared with two state-of-the-art methods.

Spectral Clustering on Spherical Coordinates Under the Degree-Corrected Stochastic Blockmodel

ABSTRACT Spectral clustering is a popular method for community detection in network graphs: starting from a matrix representation of the graph, the nodes are clustered on a low-dimensional projection obtained from a truncated spectral decomposition of the matrix. Estimating correctly the number of communities and the dimension of the reduced latent space is critical for good performance of spectral clustering algorithms. Furthermore, many real-world graphs, such as enterprise computer networks studied in cyber-security applications, often display heterogeneous within-community degree distributions. Such heterogeneous degree distributions are usually not well captured by standard spectral clustering algorithms. In this article, a novel spectral clustering algorithm is proposed for community detection under the degree-corrected stochastic blockmodel. The proposed method is based on a transformation of the spectral embedding to spherical coordinates, and a novel modeling assumption in the transformed space. The method allows for simultaneous and automated selection of the number of communities and the latent dimension for spectral embeddings of graphs with uneven node degrees. Results show improved performance over competing methods in representing computer networks.

Search for statistically significant clusters in networks and biomedical applications

A new, statistically based method of clustering previously used by authors for analysis of frequency and point data in epidemiology and for searching clusters in samples is applied to discovering clusters in graphs or networks. Graph clustering or community detection is often used in biomedical applications. Nevertheless methods existing in literature have no sufficient statistical background and the question of significance for revealed clusters usually is not considered. In the case when clusters do not form the whole network false positive results are inevitable. We propose method for discovering statistically significant clusters. It is compared with nine methods existing in literature. It is shown that only proposed method can discover significant clusters successfully discriminating them from phone consisting from randomly connected network objects.

Identification of collective particle motion in a rotating drum using a graph community detection algorithm

The discrete element method (DEM) is the method of choice in many cases of simulation and analysis of processes in granular matter, providing the data at the level of individual particles. In most applications and experiments, however, the bulk behavior is of interest; therefore, larger scale structures must be identified from the DEM simulation data. We present the method for detection of particle groups involved in collective motion based on network analysis. Knowing the positions and velocities of individual particles, a “velocity similarity graph” is built, where the graph vertices represent the particles. The vertex pairs are connected by the edge if the distance between the respective particles is small enough. The edge weight is calculated to be inversely proportional to the difference in the respective particle velocities, that is, the vertex pairs representing nearby particles having similar velocities are connected by edges of larger weight. If a group of particles moves in a coordinated matter, the particles in this group will have similar velocities; therefore, the corresponding vertices in the graph will be connected by edges of larger weight in the representing graph. Having produced the velocity similarity graph, identification of particle groups becomes equivalent to the problem of “community detection” in graph analysis. The algorithms and techniques developed for community detection in graphs can be thereby applied for identification of particle groups involved in coordinated motion in granular matter. We illustrate this approach by an example of granular media filled in a rotating cylinder.

{text{COSNet}}_i: ComplexOme-Structural Network Interpreter used to study spatial enrichment in metazoan ribosomes

BackgroundUpon environmental stimuli, ribosomes are surmised to undergo compositional rearrangements due to abundance changes among proteins assembled into the complex, leading to modulated structural and functional characteristics. Here, we present the ComplexOme-Structural Network Interpreter ({text{COSNet}}_i), a computational method to allow testing whether ribosomal proteins (rProteins) that exhibit abundance changes under specific conditions are spatially confined to particular regions within the large ribosomal complex.Results{text{COSNet}}_i translates experimentally determined structures into graphs, with nodes representing proteins and edges the spatial proximity between them. In its first implementation, {text{COSNet}}_i considers rProteins and ignores rRNA and other objects. Spatial regions are defined using a random walk with restart methodology, followed by a procedure to obtain a minimum set of regions that cover all proteins in the complex. Structural coherence is achieved by applying weights to the edges reflecting the physical proximity between purportedly contacting proteins. The weighting probabilistically guides the random-walk path trajectory. Parameter tuning during region selection provides the option to tailor the method to specific biological questions by yielding regions of different sizes with minimum overlaps. In addition, other graph community detection algorithms may be used for the {text{COSNet}}_i workflow, considering that they yield different sized, non-overlapping regions. All tested algorithms result in the same node kernels under equivalent regions. Based on the defined regions, available abundance change information of proteins is mapped onto the graph and subsequently tested for enrichment in any of the defined spatial regions. We applied {text{COSNet}}_i to the cytosolic ribosome structures of Saccharomyces cerevisiae, Oryctolagus cuniculus, and Triticum aestivum using datasets with available quantitative protein abundance change information. We found that in yeast, substoichiometric rProteins depleted from translating polysomes are significantly constrained to a ribosomal region close to the tRNA entry and exit sites.Conclusions{text{COSNet}}_i offers a computational method to partition multi-protein complexes into structural regions and a statistical approach to test for spatial enrichments of any given subsets of proteins. {text{COSNet}}_i is applicable to any multi-protein complex given appropriate structural and abundance-change data. {text{COSNet}}_i is publicly available as a GitHub repository https://github.com/MSeidelFed/COSNet_i and can be installed using the python installer pip.

Nested Stochastic Block Models applied to the analysis of single cell data

Single cell profiling has been proven to be a powerful tool in molecular biology to understand the complex behaviours of heterogeneous system. The definition of the properties of single cells is the primary endpoint of such analysis, cells are typically clustered to underpin the common determinants that can be used to describe functional properties of the cell mixture under investigation. Several approaches have been proposed to identify cell clusters; while this is matter of active research, one popular approach is based on community detection in neighbourhood graphs by optimisation of modularity. In this paper we propose an alternative and principled solution to this problem, based on Stochastic Block Models. We show that such approach not only is suitable for identification of cell groups, it also provides a solid framework to perform other relevant tasks in single cell analysis, such as label transfer. To encourage the use of Stochastic Block Models, we developed a python library, schist, that is compatible with the popular scanpy framework.

Non-convex exact community recovery in stochastic block model

Community detection in graphs that are generated according to stochastic block models (SBMs) has received much attention lately. In this paper, we focus on the binary symmetric SBM—in which a graph of n vertices is randomly generated by first partitioning the vertices into two equal-sized communities and then connecting each pair of vertices with probability that depends on their community memberships—and study the associated exact community recovery problem. Although the maximum-likelihood formulation of the problem is non-convex and discrete, we propose to tackle it using a popular iterative method called projected power iterations. To ensure fast convergence of the method, we initialize it using a point that is generated by another iterative method called orthogonal iterations, which is a classic method for computing invariant subspaces of a symmetric matrix. We show that in the logarithmic sparsity regime of the problem, with high probability the proposed two-stage method can exactly recover the two communities down to the information-theoretic limit in \({\mathcal {O}}(n\log ^2n/\log \log n)\) time, which is competitive with a host of existing state-of-the-art methods that have the same recovery performance. We also conduct numerical experiments on both synthetic and real data sets to demonstrate the efficacy of our proposed method and complement our theoretical development.